Hearing by Eyes Thanks to the “labiophone”: Exchanging Speech Movements

We present here the “labiophone”, a virtual system for audio-visual speech communication. A clone of the speaker is animated at distance by articulatory movements extracted from the speaker’s image and captured thanks to a video-camera centered on the speaker’s face. The clone consists of a mesh driven by a few articulatory parameters and clothed by blended textures. The characteristics of the articulatory model and the textures blending are transmitted at the initiation of the dialog. Then only articulatory parameters are transmitted at a very low bit rate through the telecommunication or web network. Preliminary evaluation of such a system is presented below.